Testing theories of skill learning using a very large sample of online game players

We analyse data from a very large (n = 854064) sample of players of an online game involving rapid perception, decisionmaking and motor responding. This data set allows us to connect full details of training history with measures of performance, for participants who are engaged for a sustained amount of time in effortful practice. We show that lawful relations exist between practice amount and subsequent performance, and between practice spacing and subsequent performance. This confirms results long established in the literature on skill acquisition. Additionally, we show that higher initial variation in performance is linked to subsequent higher performance, a result we link to the exploration-exploitation tradeoff from the computational framework of reinforcement learning. We discuss the benefits and opportunities of behavioural datasets with very large sample sizes and suggest that this approach could be particularly fecund for studies of skill acquisition.